SoloLab icon

Solar Orbiter Data Analysis Toolkit

Overview

This toolkit provides a comprehensive set of tools for loading, processing, analyzing, and visualizing data from the Solar Orbiter mission, primarily focusing on the STIX, RPW, and EPD instruments. It supports advanced workflows including power-law analysis, event statistics, frequency drift fitting, quicklook plotting, and file/data availability checking. The package is designed for scientific users analyzing solar flare and radio burst data, with both scripting and interactive GUI (Qt-based) workflows.

Key features:

Table of Contents

Architecture Overview

flowchart TD
  subgraph DataPreparation["Data Preparation"]
        stixRead[stix_read.py]
        rpwRead[rpw_read.py]
        values[values.py]
        fileAvailability[file_availability.py]
    end
  subgraph Analysis["Analysis and Fitting"]
        electronPowerlaw[electron_powerlaw.py]
        fitFunctions[fit_functions.py]
        freqDrift[freq_drift.py]
        flareStatistics[flare_statistics.py]
    end
  subgraph Visualization["Visualization and Quicklooks"]
        quicklooks[quicklooks.py]
    end
  subgraph GUI["Graphical User Interface"]
        sololabApp[sololab_app.py]
    end
    stixRead --> electronPowerlaw
    stixRead --> quicklooks
    rpwRead --> freqDrift
    rpwRead --> quicklooks
    electronPowerlaw --> quicklooks
    fitFunctions --> freqDrift
    freqDrift --> quicklooks
    flareStatistics --> quicklooks
    values --> stixRead
    values --> rpwRead
    values --> fitFunctions
    values --> quicklooks
    values --> flareStatistics
    sololabApp --> quicklooks
    sololabApp --> stixRead
    sololabApp --> rpwRead
    sololabApp --> values

Module Structure and Responsibilities

Module/File Responsibility
stix_read.py Reading and preprocessing STIX L1/L2 FITS files, background subtraction, file merging
rpw_read.py Reading, filtering, and preprocessing RPW HFR/TNR CDF data, background subtraction, PSD creation, and combination
values.py Constants, date/time formats, utility functions (mean, filtering) used throughout the codebase
file_availability.py File system utilities for data/bkg file discovery, time interval checking, suggestions for background files
quicklooks.py High-level plotting utilities for quicklooks, combined views, overlays, and publication-ready instrument plots
sololab_app.py Qt-based GUI application for data import, interactive parameter selection, and combined plotting
freq_drift.py Fitting time-frequency profiles, extracting exciter velocities and drift rates, and diagnostics for frequency drift phenomena
flare_statistics.py CSV-based flare event table parsing, calculation of delays, heights, and expected parameter estimates
electron_powerlaw.py Electron flux modeling, power-law and double power-law electron distributions, STIX data integration, plotting
fit_functions.py Mathematical model definitions (Gaussian, Weibull, fluorescence, etc.) for scientific fitting and their error metrics
__init__.py Package-wide import aggregator, exposes all public functions/classes for interactive use

Detailed Module Descriptions

A. Read & Core Utility Modules

1. STIX Data Reading (stix_read.py)

Purpose

Handles loading and preprocessing of STIX L1/L2 FITS files, including counts extraction, time/energy bin normalization, background subtraction, time selection, and merging of multiple files.

Main Functions

Name Type Description
stix_create_counts function Reads a STIX FITS file, returns counts per second per energy bin, and time array.
stix_remove_bkg_counts function Subtracts background using another file or custom time interval, with optional polling function.
stix_combine_files function Merges multiple count files, optionally with background or energy alignment enforcement.
stix_combine_counts function Merges multiple in-memory count objects, aligning on time and energy bins.

Key Behavior:

2. RPW Data Reading (rpw_read.py)

Purpose

Handles reading and preprocessing of RPW HFR/TNR data from CDF files, PSD computation, background subtraction, and combination of PSDs across files.

Main Functions

Name Type Description
rpw_get_data function Loads RPW L2/L3 CDF files, applies frequency filtering and pre-processing.
rpw_create_PSD function Creates a Power Spectral Density (PSD) object from loaded data, supports date/frequency selection and background subtraction.
rpw_create_PSD_L2/L3 function Level-specific PSD creation logic.
rpw_sort_psd function Ensures PSD arrays are sorted by time/frequency axes.
rpw_join_psds function Joins multiple compatible PSDs into one (with background aggregation).

Key Behavior:

3. Values, Constants, and Utilities (values.py)

Purpose

Centralizes all constants for physical units, date/time formats, RPW/STIX index arrays, and common utility functions (e.g., geometric mean, polling functions, filtering NaNs).

Key Features

4. File Availability Utilities (file_availability.py)

Purpose

Provides file system tools for scanning directories for STIX/RPW science and background files, checking data availability for date intervals, and providing suggestions for background file selection.

Main Functions

Name Type Description
stix_get_all_bkg_files function Lists all STIX background FITS files in a directory.
stix_get_all_aux_files function Lists all STIX auxiliary files.
stix_suggest_bkg_file_for_date function Suggests closest-in-time background files to a given date.
stix_data_in_interval_exists function Checks if science files cover a given interval.
rpw_check_date_availability function Checks if RPW CDF files cover a given date.
check_combined_availability function Reports combined availability for STIX and RPW data.
save_objects, load_objects function Simple save/load for numpy .npz instrument data packs.

B. Visualization & App Modules

5. Quicklook and Combined Plotting (quicklooks.py)

Purpose

Centralizes all Matplotlib-based visualization routines for quicklooks, overlays, and publication-ready plots for STIX, RPW, and EPD data, including joint and multi-panel displays.

Main Functions

Name Type Description
quicklook_plot function High-level function to generate combined instrument plots with many customization options.
stix_plot_spectrogram, stix_plot_counts, stix_plot_overlay function STIX-specific plotting routines (spectrogram, integrated counts, overlays).
stix_plot_bkg function Plots STIX background spectrum (counts/s/keV) versus mean energy from a background-corrected STIX counts object.
rpw_plot_psd, rpw_plot_curves, rpw_plot_overlay function RPW-specific plotting routines (PSD, integrated curves, overlays).
rpw_plot_bkg function Plots selected RPW background spectrum versus frequency and overlays per-frequency data maxima.
rpw_plot_bkg_aggregation function For merged RPW PSDs, plots each component background plus the aggregated background used in the combined PSD.
paint_markers, ax_paint_grid function Auxiliary functions for plot annotation and formatting.
plot_ept_data function Plots EPD (particle) time profiles for selected channels.

Key Behavior:

6. Qt Application (sololab_app.py)

Purpose

Implements a full-featured Qt (via qtpy) GUI for data import, parameter selection, and interactive plotting. Supports instrument selection, data pack saving/loading, and detailed plotting preferences.

Main Classes

Class/File Description
MainWindow Main application window, manages state and orchestrates dialogs.
ImportStixDialog Dialog for loading and previewing STIX data (with background options).
ImportRpwHfrDialog, ImportRpwTnrDialog Dialogs for loading and previewing RPW HFR/TNR data (with background options).
ImportEpdDialog Dialog for downloading and previewing EPD particle data.
PlotPrefsDialog Dialog for setting and saving plotting preferences (energy/frequency bins, log-scaling, overlays, etc.).
CombinedPlotDialog Dialog for selecting displayed instruments and combined figure plotting.
InstrumentSelectionDialog, EpdChannelSelectionDialog, etc. Utility dialogs for instrument/channel/frequency selection.

GUI Features

Example GUI Flow

sequenceDiagram
    participant User
    participant GUI as MainWindow
    participant Importer as ImportStixDialog/ImportRpwDialog
    participant Plotter as quicklook_plot
    User->>GUI: Click "Import STIX data"
    GUI->>Importer: Show file selection dialog
    Importer->>GUI: Returns processed data
    GUI->>Plotter: Generate preview/quicklook
    Plotter-->>GUI: Returns Matplotlib figure
    GUI-->>User: Displays plot in preview area

C. Advanced Analysis Modules

7. Frequency Drift Analysis (freq_drift.py)

Purpose

Provides the core algorithms for fitting time-frequency profiles (e.g., type III radio burst drift), extracting drift rates, exciter velocities, and plotting diagnostic results.

Main Functions

Name Type Description
rpw_fit_time_profiles function Fits temporal profiles for all frequency bins using specified model (Gaussian, Weibull, etc.).
rpw_freq_drifts function Calculates frequency drift rates and uncertainties from fit results.
rpw_estimate_beam_velocity function Estimates exciter/beam velocities from drift rates with error propagation using density models.
plot_compare_fits, plot_curve_fits, etc. function Visualization routines for comparing fits, plotting fit overlays, and velocity diagnostics.
ne_from_freq, freq_from_ne, ne_from_r_leblanc function Electron density model utilities for coronal/heliospheric calculations.

Key Behavior:

8. Flare Statistics (flare_statistics.py)

Purpose

Parses, computes, and augments flare event tables (CSV format) with derived delay times, onset/peak events, and physical expectations based on density models and velocities.

Main Functions

Name Type Description
create_dataframe_from_csv function Loads a flare event table, computes delays between various event times, filters by 'used' flag.
estimate_expected_heights_delays function Adds columns for expected heights and delays for given velocity values and electron density models.

9. Electron Powerlaw Modeling (electron_powerlaw.py)

Purpose

Provides classes and functions for representing and manipulating electron flux distributions as power-law or double power-law models, including integration with STIX spectroscopy data. Supports extrapolation, integration, and various plotting utilities.

Main Classes & Functions

Name Type Description
powerlaw class Models a single power-law electron spectrum with extrapolation, integration, and plotting.
double_powerlaw class Models a broken/double power-law electron spectrum (two spectral indices, one break energy).
flux_e_low_from_total_flux function Computes flux at low energy from total flux and spectral index.
plot_powerlaws function Plots multiple powerlaw/double_powerlaw instances in a combined quicklook.
process_spectroscopy_table function Processes a STIX spectroscopy result table, extracts key fit parameters, and computes integrated fluxes above energy thresholds.

Key Behavior:

10. Fitting Functions (fit_functions.py)

Purpose

Defines a set of parametric models (Gaussian, Weibull, etc.) and their corresponding error/metric functions for fitting intensity/time profiles in solar data. Centralizes all mathematical models for reuse across analysis routines.

Main Classes & Functions

Name Type Description
intensity_model class Encapsulates a fit function, its center, deviation, background, onset, and error calculation methods.
FUNCTION_CATALOG dict Maps model names to their intensity_model instances for lookup by string.
get_fit_function function Retrieves a model from the catalog by name.
get_metric function Returns metric (e.g., RMSE, weighted RMSE) for fit evaluation.
gaussian, fluorescence, weibull, etc. function Mathematical model functions for use in fitting.

Key Behavior:

11. Package Init (__init__.py)

Purpose

Aggregates all main functions and classes from submodules to provide a flat package namespace for interactive and script imports.

Key Classes Reference

Class Location Responsibility
powerlaw, double_powerlaw electron_powerlaw.py Electron flux modeling and integration
intensity_model fit_functions.py Encapsulates parametric fit functions
solar_event quicklooks.py Event annotation for time series plots
MainWindow sololab_app.py Main Qt application class
ImportStixDialog sololab_app.py STIX data import dialog
ImportRpwHfrDialog sololab_app.py RPW HFR data import dialog
ImportEpdDialog sololab_app.py EPD data import and download dialog

Error Handling

Caching Strategy

Dependencies

Integration Points

Testing Considerations

Parameter and Object Format Reference

This section adds practical typing/context for the most used functions. Types below use Python-style notation.

Global date/time conventions

Core object schemas used across modules

STIX counts object (dict)

Returned by stix_create_counts and related helpers:

RPW raw data object (dict)

Returned by rpw_get_data:

RPW PSD object (dict)

Returned by rpw_create_PSD*:

STIX API: expected parameters

stix_create_counts(pathfile, is_bkg=False, time_arr=None, date_range=None, correct_flight_time=False, energy_shift=0)

stix_remove_bkg_counts(pathfile, pathbkg=None, stix_bkg_range=None, date_range=None, ..., bkg_poll_function='mean')

stix_combine_files(filenames, bkgfile=None, ..., stix_bkg_range=None, force=False, ...)

RPW API: expected parameters

rpw_get_data(file, sensor=None, filter=tnr_remove_idx)

rpw_create_PSD(data, freq_range=None, date_range=None, freq_col=0, proposed_indexes=..., which_freqs='both', rpw_bkg_interval=None, sort=True, bkg_poll_function='mean')

rpw_join_psds(psds, bkg_agg='median', plot_bkg_agg=False)

Quicklook plotting API: expected parameters

quicklook_plot(...)

Most important inputs:

Common quicklook helper object expectations

Background plotting helper inputs

Frequency-drift fitting API: expected parameters

rpw_fit_time_profiles(rpw_psd, func, time_range, frequency_range, metrics=['rmse'], excluded_freqs=[], tol=0.2, smooth_pts=None, secondary_loop=False, name_tag='')

Density/conversion utility inputs (freq_drift.py)

Flare statistics and table-processing expectations

create_dataframe_from_csv(path, remove_not_used=True)

estimate_expected_heights_delays(dataframe, density_model=..., name='leblanc', velocities=[...])

Electron power-law API: expected parameters

flux_e_low_from_total_flux(delta, total, e_low)

plot_powerlaws(pwlws, e_range, above=False, logx=False, colors=None, styles=None)

process_spectroscopy_table(specdata_raw, energy_thresholds=[...], include=['double','single'], time_phase=0)

Example API Usage (No HTTP endpoints found)

This package does not expose HTTP REST API endpoints. All data loading, analysis, and plotting are conducted in local Python processes or via the interactive GUI.

Summary

This Solar Orbiter Data Analysis Toolkit centralizes all major workflows for multi-instrument solar flare and type III burst analysis, supporting both automated analysis pipelines and interactive exploration via a robust Qt GUI. The codebase is modular, extensible, and uses standard scientific Python libraries for maximum compatibility.

{
  "title": "Data Consistency Checks",
  "content": "All data merging functions align on energy/frequency bins and check for compatibility, issuing warnings when bins are mismatched."
}

{
  "title": "Error Propagation",
  "content": "Fit routines compute and propagate parameter uncertainties, supporting robust physical inference from event data."
}

{
  "title": "GUI Data Pack Save/Load",
  "content": "Users can save all imported and processed data to a pickled pack for reproducibility and quick reloads."
}